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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In the Patent Application of 

Zheng J. Geng 

Application No. 09/821,648 

Filed: March 29, 2001 

For: Method and Apparatus for 
Omnidirectional Imaging 



Group Art Unit: 2621 
Examiner: REKSTAD, Erick J. 



APPEAL BRIEF 

Mail Stop Appeal Brief - Patents 
Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 

Sir: 

The Appeal Brief filed herewith is in response to the Notification of Non-Compliant 
Appeal Brief mailed on April 3, 2007. This is an Appeal Brief under Rule 41.37 appealing 
the final decision of the Primary Examiner dated September 21, 2005. Each of the topics 
required by Rule 41.37 is presented herewith and is labeled appropriately. 
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APR 2 5 2007 

L Real Party in Interest 

The real party in interest is Genex Technologies, Inc. of Maryland, The inventor has 
assigned all rights in the invention and the present application to Genex Technologies, Inc. 



II. Related Appeals and Interferences 
There are no appeals or interferences related to the present application of which the 
Appellants are aware. 



IIL Status of Claims 

Claims 1-14 and 16-44 are currently pending in the application and all stand finally 
rejected. Claim 15 has been cancelled. Appellant appeals from the final rejection of claims 
1-14 and 16-44, which claims are presented in the Appendix. 



IV. Status of Amendments 
On November 21, 2005, Appellant filed one response to the final Office Action 
mailed on September 21, 2005. However, that response made no amendments to the 
application. Consequently, its entry into the record has no impact on the content of the claims 
presented on appeal. 



V- Snmmary of Claimed Subject Matter 

A number of approaches had been proposed for imaging systems that attempt to 
achieve a wide field-of-view (FOV). (Appellant's specification, paragraph 0003). Wide- 
viewing-angle lens systems, such as fish-eye lenses, are designed to have a very short focal 
length which, when used in place of conventional camera lens, enables the camera to view 
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objects at much wider angle to obtain a panoramic view, as shown in Figure 1. In general, to 
widen the FOV, the design of the fish-eye lens is made more complicated. Moreover, tjie 
nonlinear properties resulting from the semi-spherical optical lens mapping make the 
resolution along the circular boundary of the image very poor. Although the images acquired 
by fish-eye lenses may be adequate for certain low-precision visualization applications, these 
lenses still do not provide adequate distortion compensation. (Appellant's specification, 
paragraph 0004). 

To dramatically increase the field of view of an imaging system, Appellant employs a 
reflective surface to obtain an omnidirectional image. In particular, the field of view of a 
video camera can be greatly increased by using a reflective surface with a properly designed 
surface shape that provides a greater field of view than a flat reflective surface. There are a 
number of surface profiles that can be used to produce an omnidirectional FOV. (Appellant's 
specification, paragraph 0025). 

Figures 2a through 2c appear to indicate that any convex mirror can be used for 
omnidirectional imaging; however, a satisfactory imaging system must meet two 
requirements. First, the system must create a one-to-one geometric correspondence between 
pixels in an image and points in the scene. Second, the convex mirror should conform to a 
"single viewpoint constraint;" that is, each pixel in the image corresponds to a particular 
viewing direction defined by a ray from that pixel on an image plane through a single viewing 
point such that all of the light rays are directed to a single virtual viewing point. Based on 
these two requirements, the convex mirrors shown in Figures 2a through 2c can increase the 
field of view but are not satisfactory imaging devices because the reflecting surfaces of the 
mirrors do not meet the single viewpoint constraint, which is desirable for a high-quality 
omnidirectional imaging system. (Appellant's specification, paragraph 0026). 
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The preferred design for a reflective surface is described with reference to Figure 3. 
Figure 3 shows a video camera 30 having an image plane 3 1 on which images are captured 
and a regular lens 32 whose field of view preferably covers the entire reflecting surface of the 
mirror 34. Since the optical design of camera 30 and lens 32 is rotationally symmetric, only 
the cross-sectional function z(r) defining the mirror surface cross-section profile needs to be 
determined. The actual mirror shape is generated by the revolution of the desired cross* 
section profile about its optical axis. The function of the mirror 34 is to reflect all viewing 
rays coming from the video camera's 30 focal point C to the surface of physical objects in the 
field of view. A hyperbola is the preferred cross-sectional shape of the mirror 34 because a 
hyperbolic mirror will satisfy the geometric correspondence and single viewpoint constraint 
requirements of the system, (Appellant's specification, paragraph 0027-8). 

The image obtained by the camera 30 and capture on the camera's image plane 31 will 
exhibit some distortion due to the non-planar reflecting surface of the mirror 34. To facilitate 
the real-time processing of the omnidirectional image, the inventive system uses an algorithm 
to map the pixels from the distorted omnidirectional image on the camera's image plane 31 
onto a perspective window image 40 directly, once the configuration of the perspective or 
panoramic window is defined. As shown in Figure 4, a virtual perspective viewing window 
40 can be arbitrarily defined in a three-dimensional space using three parameters: Zoom, Pan 
and Tilt- All of these parameters can be adjusted by the user. (Appellant's specification, 
paragraph 0030). 

Once the perspective viewing window W 40 is defined, the system can establish a 
mapping matrix that relates the pixels in the distorted omnidirectional image I(i j) to pixels 
W(p,q) in the user-defined perspective viewing window W 40 to form a non-distorted 
perspective image. The conversion from the distorted omnidirectional image into a non- 
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distorted perspective image using a one-to-one pixel correspondence between the two images 
is unique. (Appellant's specification, paragraph 003 1), 

Once the mapping matrix MAP has been established, the real-time image-processing 
task is greatly simplified and can be conducted in a single step at step 70 by applying the 
mapping matrix MAP to each pixel I(i j) in the omnidirectional image I to determine the pixel 
values for each corresponding pixel m the perspective viewing window W. Further, each 
time a new omnidirectional image I is acquired, a look-up table operation can be performed to 
generate the non-distorted perspective image for display in the perspective viewing window 
W at step 72. (Appellant's specification, paragraph 0036). 

Turning to Appellant's independent claims: 

1 . A method for generating a selectable perspective view of a portion of a 
hemispherical image scene, comprising the steps of: 

acquiring an omnidirectional image on an image plane (31) using a reflective mirror 
(34) that satisfies a single viewpoint constraint and an image sensor (30) (Appellant's 
specification, paragraph 0025); 

defining a perspective viewing window (62) based on configuration parameters 
(Appellant's specification, paragraph 0030)\ and 

mapping (68) each pixel in the perspective window with a corresponding pixel value 
in the omnidirectional image on the image plane using a look-up table based on the 
configuration parameters (Appellant p s specification* paragraph 0036). 

14. An improved imaging apparatus for generating a two-dimensional image, 
comprising: 

5 
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a reflective mirror (34) configured to satisfy an optical single viewpoint constraint for 
reflecting an image scene {Appellant's specification, paragraph 0025); 

an image sensor (30) responsive to said reflective mirror (34) and that generates two 
dimensional image data signals to obtain an omnidirectional image on an image plane (31) 
(Appellant's specification, paragraph 0040); and 

a controller (120) coupled to the image sensor (30), wherein the controller defines 
(62) a perspective viewing window based on configuration parameters and maps pixels (68) 
from said omnidirectional image into said perspective viewing window (Appellant's 
specification, paragraph 003 S); and 

a memory (120) for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller using a said mapping matrix to 
perform mapping of pixels from said omnidirectional image into said perspective viewing 
window (Appellant's specification, paragraph 0031). 

31. An imaging apparatus for generating a two-dimensional image, comprising: 

a reflective hyperbolic mirror (34) having a hyperbolic cross-section (Appellant f s 
specification, paragraph 0028); 

an image sensor (30) optically coupled to said reflective mirror (34) that generates 
two-dimensional image data signals based on an omnidirectional image reflected by said 
mirror (34) (Appellant 's specification, paragraph 0040); and 

a controller (120) coupled to the image sensor (30), wherein the controller (120) 
defines a perspective viewing window based on configuration parameters and maps pixels 
from said omnidirectional image into said perspective viewing window (Appellant 's 
specification, paragraph 0036)\ and 
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a memory (120) for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller (120) using a said mapping 
matrix to perform mapping of pixels from said omnidirectional image into said perspective 
viewing window {Appellant's specification, paragraph 0031). 

VI. Grounds of Refection to be Reviewed on Appeal 

In the final Office Action, the following rejections were made for which review is 
requested by this appeal. 

(1) Claims 1-6 were rejected as unpatentable under 35 U.S.C. § 103(a) in view of the 
combined teachings of U.S. Patent No. 6,1 18,474 to Nayar ( 4e Nayar") and U.S. Patent No. 
5,870,135 to Glatt et at ("Glatt"). 

(2) Claims 14 and 16-23 were rejected as being unpatentable under 35 U,S.C § 103(a) 
in view of the combined teachings of Nayar and U.S. Patent No. 4,908,874 to Gabriel 
("Gabriel"). 

(3) Claims 31-38 and 44 were rejected as unpatentable under 35 U.S.C § 103(a) over 
the combined teachings of Nayar> Gabriel and U.S. Patent No., 6,226,035 to Korein et al 
("Korein"). 

(4) Dependent claims 7-9 were rejected under 35 U.S.C. § 103(a) over the combined 
teachings of Nayar and U.S. Patent No. 5,790,181 to Chahl et al ("Chain 1 ") and U.S. Patent 
No. 3,988,533 to Mick et al. ("Mick")- 

(5) Dependent claims 10-13 were rejected under 35 U.S.C. § 103(a) over the 
combined teachings of Nayar, Glatt and U.S. Patent No. 5.686,975 to Baker ("Baker"). 

(6) Dependent claims 24 and 25 were rejected under 35 U.S.C. § 103(a) over the 
combined teachings of Nayar, Gabriel, Chahl and Mick- 
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(7) Dependent claims 26-29 were rejected under 35 U.S.C. § 103(a) over the 
combined teachings of Nayar, Gabriel, and Baker. 

(8) Dependent claim 39 and 40 were rejected as unpatentable under 35 U.S.C § 
103(a) over the combined teachings of Nayar, Gabriel, Korein, Chahl and Mick. 

(9) Dependent claim 41-43 were rejected as unpatentable under 35 U.S.C. § 103(a) 
over the combined teachings of Nayar, Gabriel, Korein, and Baker, 

VII. Argument 

Claim 1: 

Claims 1 -6 were rejected as unpatentable under 35 U,S-C § 103(a) in view of the 
combined teachings of U.S. Patent No. 6,1 18,474 to Nayar 0*Nayar") and U.S. Patent No. 
5,870,135 to Glatt et aL ("GlatO- 

Claim 1 recites: 

A method for generating a selectable perspective view of a portion of a 
hemispherical image scene, comprising the steps of: 

acquiring an omnidirectional image on an image plane using a reflective 
mirror that satisfies a single viewpoint constraint and an image sensor; 

defining a perspective viewing window based on configuration parameters; 

and 

mapping each pixel in the perspective window with a corresponding pixel 
value in the omnidirectional image on the image plane using a look-up table based on 
the configuration parameters. 

Nayar is cited because Nayar teaches the collection of a wide-angle image using 
a truncated, substantially paraboloid-shaped reflector." (Nayar, abstract). However, in 
/contrast to claim 1, Nayar fails to teach or suggest the claimed mapping of pixels from an 
omnidirectional image to a perspective viewing window "using a look-up table." The recent 
Office Action concedes this point (Action of 9/21/05, p. 2, 6t Nayar is lacking the use of look- 
up tables for the mapping.") 
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Consequently, the Office Action proposed to combine the teachings of Glatt with 
Nayar. According to the Office Action, "Glatt teaches the mapping from a fish-eye lens 
(which is hemispherical) to cartesian coordinates using a look-up table." (Action of 9/21/05, 
p. 2). Glatt describes in detail the equations that are used to map the image from the fish-eye 
lens. (Glatt, col. 7, line 46 to col 8> line 43). Glatt, however, does not teach or suggest how 
mapping would be performed using a look-up table for an image that comes, not from a fish- 
eye lens, but from a reflective mirror as claimed. 

The mapping function of Glatt cannot be imported into the system taught by Nayar 
because Nayar does not operate on an image produced with a fish-eye lens. The vast 
differences between the use of a mirror and a fish-eye lens and the problems associated with 
the use of a fi$h-eye lens are documented in Appellant's specification at, for example, 
paragraph 0004 as noted in the Summary of Claimed Subject Matter above.. 

"An analysis of obviousness of a claimed combination must include consideration of 
the results achieved by that combination." Gillette Co, v, S.G Johnson & Son, Inc., 919 F.2d 
720 (Fed. Cir. 1990), "If the proposed modification or combination of the prior art would 
change the principle of operation of the prior art invention being modified, then the teachings 
of the references are not sufficient to render the claims prima facie obvious. In re Ratti^ 270 
F.2d 810, 123 USPQ 349 (CCPA 1959)." M.P.E.P. § 2143.QL 

More importantly, n [i]n order to render a claimed apparatus or method obvious, the 
prior art must enable one skilled in the art to make and use the apparatus or method. H 
Beckman Instruments, Inc. v. LKB Produkter AB, 892 F,2d 1547, 1551, 13 U.S.P.Q.2d 1301, 
1304 (Fed. Cir. 1989); In re Payne, 606 F,2d 303, 314, 203 US JP.Q. 245, 255 (CCPA 1979). 
In the present case, the teachings of Nayar and Glatt, even if combined, do not enable one of 
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skill in the art to make and use the claimed method in which a look-up table is used to map an 
image from a reflective mirror, rather than from a fish-eye lens. 

For any and all of these reasons, the teachings of Nayar and Glatt fail to enable or 
render obvious the subject matter of claim 1. Therefore, the rejection of claim 1 and its 
dependent claims based on Nayar and Glatt should not be sustained. 



Claim 2, 3^ 18. 19. 34 and 35: 

The various dependent claims in the application recite further subject matter that is 

not taught or suggested by the prior art of record. For example,, claim 2 recites: 

wherein the configuration parameters defined in the defining step include at least one 
of a zoom distance defined as the distance from the focal point of said reflective 
mirror to said window, a pan angle defined as the angle between the x axis and a line 
through the focal point of said reflective mirror perpendicular to the x-y plane and a 
tilt angle defined as the angle between the x-y plane and a vector normal to said 
window. 

Claims 18 and 34 recite similar subject matter. 

In contrast, Nayar and Glatt fail to teach or suggest the claimed configuration 
parameters including a zoom distance defined as the distance from the focal point of a 
reflective mirror to a user-defmed window, a pan angle and tilt angle as claimed. For at least 
this additional reason, the rejection of claims 2, 3, 18, 19, 34 and 35 should not be sustained. 



Claims 4., 5, 20 and 36: 
Claim 4 recites: 

wherein the mapping step includes the step of generating a mapping matrix by: 
applying a ray tracing algorithm to each pixel in the perspective viewing 

window to determine a corresponding reflection point on the reflective mirror; and 
projecting each reflection point to a focal point of the image sensor to 

determine the corresponding location in the omnidirectional image on the image 

plane. 

10 
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Claims 20 and 36 recite similar subject matter. 

In contrast, Nayar and Glatt, as demonstrated above, teach mapping from a fish-eye 
lens. Consequently, Nayar and Glatt cannot and do not teach or suggest a method having a 
mapping step that includes applying a ray tracing algorithm to each pixel in the perspective 
viewing window to determine a corresponding reflection point on the reflective mirror as 
claimed. For at least this additional reason, the rejection of claims 4, 5, 30 and 36 should not 
be sustained. 



Claims 14, 16-23. 31-38 and 44: 

Claims 14, 16-23, 31-38 and 44 were rejected as being unpatentable under 35 U.S.C. 
§ 103(a) in view of the combined teachings of Nayar and U.S. Patent No. 4,908,874 to 
Gabriel ("Gabriel"). For at least the following reasons, this rejection is respectfully traversed. 

Claim 14 recites: 

An improved imaging apparatus for generating a two-dimensional image, 
comprising: 

a reflective mirror configured to satisfy an optical single viewpoint constraint 
for reflecting an image scene; 

an image sensor responsive to said reflective mirror and that generates two 
dimensional image data signals to obtain an omnidirectional image on an image plane; 
and 

a controller coupled to the image sensor, wherein the controller defines a 
perspective viewing window based on configuration parameters and maps pixels from 
said omnidirectional image into said perspective viewing window; and 

a memory for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller using a said mapping 
matrix to perform mapping of pixels from said omnidirectional image into said 
perspective viewing window. 
(emphasis added). 
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Claims 31-38 and 44 were rejected as unpatentable under 35 U.S.C. § 1 03(a) over the 
combined teachings of Nayar, Gabriel and U.S. Patent No., 6,226,035 to Korein et el. 

("Korein"). 

Similar to claim 14, claim 3 1 recites: 

An imaging apparatus for generating a two-dimensional image, comprising: 
a reflective hyperbolic mirror having a hyperbolic cross-section; 
an image sensor optically coupled to said reflective mirror that generates two- 
dimensional image data signals based on an omnidirectional image reflected by said 
mirror; and 

a controller coupled to the image sensor, wherein the controller defines a 
perspective viewing window based on configuration parameters and maps pixels from 
said omnidirectional image into said perspective viewing window; and 

a memory for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller using a said mapping 
matrix to perform mapping of pixels from said omnidirectional image into said 
perspective viewing window. 

As conceded in the Office Action, "Nayar does not specifically teach the memory for 
storing a mapping matrix for each of a plurality of sets of said configuration parameters in 
parameter space." (Action of 9/21/05, p. 10). Consequently, the Office Action cites to 
Gabriel, which allegedly teaches "the use of matrices to perform transformations such as 
translation, contraction, expansion rotation and perspective projection." (Id.). Applicant 
respectfully submits that even if this interpretation of Gabriel's teachings is accurate, it is 
insufficient to support a rejection of claim 14. 

Gabriel does not teach or suggest "a memory for storing a mapping matrix for each of 
a plurality of sets of said configuration parameters in a parameter space The Office 
Action fails to indicate how or where Gabriel or any other cited prior art reference teaches or 
suggests a memory storing a mapping matrix for each of a plurality of sets of configuration 
parameters* The final Office Action does not even expressly allege that Gabriel teaches this 
subject matter. 
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Moreover, the Office Action states that the teachings of Gabriel are applicable "to 
perfoxm complex warping of images such as to a fish-eye image " (Action of 9/21/05, p. 3). 
As demonstrated above, this further proves the inapplicability of the teachings of Gabriel to 
claim 14, which calls for a method using an omidirectional image acquired with a "reflective 
mirror." The teachings of Gabriel cannot be applied to a method in which mapping is 
performed using an image from a reflective mirror rather than a fish-eye lens. 

"To establish prima facie obviousness of a claimed invention, all the claim limitations 
must be taught or suggested by the prior art. In re Royka, 490 F.2d 981, 180 USPQ 580 
(CCPA 1974)." M.P.E.P. § 2143.03. Accord. M.P.E.P. § 706.02(j). For at least these 
reasons, the rejection of claim 14 and its dependent claims and claims 31-38 and 44 should 
not be sustained. 

Claim 38: 

Claim 38 recites: 'The imaging apparatus of claim 31, wherein said memory contains 
a predetermined mapping matrix for every set of configuration parameters in said parameter 
space." In contrast, the combination of Nayar, Gabriel and Korein fails to teach or suggest 
the claimed memory containing a predetermined mapping matrix for every set of 
configuration parameters in a parameter space. For at least this additional reason, the 
rejection of claim 38 should not be sustained. 
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In view of the foregoing, it is submitted that the final rejection of the pending claims 
is improper and should not be sustained. Therefore, a reversal of the Final Rejection of 
September 21, 2005 is respectfully requested. 



Respectfully submitted, 



DATE: April 25, 2007 



Steven L. Nichols, Esq. 
Managing Partner, Utah Office 
Rader Fishman & Grauer PLLC 
River Park Corporate Center One 
10653 S. River Front Parkway, Suite 150 
South Jordan, Utah 84095 
(801) 572-8066 
(801) 572-7666 (fax) 



Steven L. Nichols 
Registration No. 40,326 
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VIII. CLAIMS APPENDIX 

1 . (previously presented) A method for generating a selectable perspective view 
of a portion of a hemispherical image scene, comprising the steps of: 

acquiring an omnidirectional image on an image plane using a reflective mirror that 

satisfies a single viewpoint constraint and an image sensor; 

defining a perspective viewing window based on configuration parameters; and 
mapping each pixel in the perspective window with a corresponding pixel value in the 

omnidirectional image on the image plane using a look-up table based on the configuration 

parameters. 

2. (original) The method of claim 1 , wherein the configuration parameters 
defined in the defining step include at least one of a zoom distance defined as the distance 
from the focal point of said reflective mirror to said window, a pan angle defined as the angle 
between the x axis and a line through the focal point of said reflective mirror perpendicular to 
the x-y plane and a tilt angle defined as the angle between the x-y plane and a vector normal 
to said window. 

3. (original) The method of claim 2, wherein the defining step is conducted 
via a user interface through which a user enters data corresponding to at least one of a desired 
zoom distance, pan angle, or tilt angle. 

4. (original) The method of claim 1 , wherein the mapping step includes the 
step of generating a mapping matrix by: 
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applying a ray tracing algorithm to each pixel in the perspective viewing window to 
determine a corresponding reflection point on the reflective mirror; and 

projecting each reflection point to a focal point of the image sensor to determine the 
corresponding location in the omnidirectional image on the image plane. 

5 . (original) The method of claim 4, further comprising the step of storing 
the mapping matrix in a module having a memory. 

6. (original) The method of claim 1 wherein the step of defining a 
perspective viewing window defines the perspective viewing window as a panoramic viewing 
window. 

7. (original) The method of claim 1 , further comprising the steps of: 
calculating a residual image based on a difference between a reference 

omnidirectional image and a sequential omnidirectional image; 

determining if the residual image contains any value that exceeds a predetermined 
threshold; and 

classifying any value that exceeds the predetermined threshold as an anomaly, 

8. (original) The method of claim 7 7 further comprising the steps of: 
calculating the configuration parameters for the perspective viewing window from the 

anomaly; and 

selectively focusing the perspective viewing window on the anomaly using the 
calculated configuration parameters. 
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9. (original) The method of claim 7, further comprising the step of 
activating an alarm if at least a portion of the residual image exceeds a predetermined 
threshold. 

1 0. (original) The method of claim 1 , further comprising the steps of: 
detecting a location of a sound source in the image scene; and 

adjusting the perspective viewing window based on the detected location of the sound 

source. 

1 1 . (original) The method of claim 1 ? further comprising the step of 
transmitting the omnidirectional image via the Internet. 

12. (original) The method of claim 1 1, wherein the transmitting step is 
conducted through a server that receives the omnidirectional image and transmits the 
omnidirectional image to at least one client. 

13. (original) The method of claim 1, further comprising the step of forming a 
two-way transmission link between the image sensor and a remote display, wherein the two- 
way transmission link transmits at least one of the omnidirectional image, the perspective 
viewing window, and an audio signal. 

1 4. (previously presented) An improved imaging apparatus for generating a two- 
dimensional image, comprising: 
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a reflective mirror configured to satisfy an optical single viewpoint constraint for 
reflecting an image scene; 

an image sensor responsive to said reflective mirror and that generates two 
dimensional image data signals to obtain an omnidirectional image on an image plane; and 

a controller coupled to the image sensor, wherein the controller defines a perspective 
viewing window based on configuration parameters and maps pixels from said 
omnidirectional image into said perspective viewing window; and 

a memory for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller using a said mapping matrix to 
perform mapping of pixels from said omnidirectional image into said perspective viewing 
window. 

15. (cancelled). 

16. (original) The improved imaging apparatus of claim 14, wherein the 
reflective mirror creates a one-to-one correspondence between pixels in the omnidirectional 
image and pixels in the perspective viewing window. 

1 7. (original) The improved imaging apparatus of claim 14, wherein the 
controller maps the omnidirectional image to the perspective viewing window by mapping 
each pixel in the perspective viewing window with a corresponding pixel value in the 
omnidirectional image. 
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18. (original) The improved imaging apparatus of claim 14, wherein the 
parameters defining the perspective viewing window include at least one of a zoom distance 
defined as the distance from the focal point of said reflective mirror to said window, a pan 
angle defined as the angle between the x axis and a line through the focal point of said 
reflective mirror perpendicular to the x-y plane and a tilt angle defined as the angle between 
the x-y plane and a vector normal to the perspective viewing window. 

19. (original) The unproved imaging apparatus of claim 1 8, further 
comprising a user interface through which a user enters data corresponding to at least one of a 
desired zoom distance, pan angle, or tilt angle. 

20- (previously presented) The improved imaging apparatus of claim 14, wherein 
the controller generates a mapping matrix by applying a ray tracing algorithm to each pixel in 
the perspective viewing window to determine a corresponding reflection point on the 
reflective mirror and then projecting each reflection point to a focal point of the image sensor 
to determine the corresponding location on the omnidirectional image. 

21. (original) The improved imaging apparatus of claim 14, wherein the 
perspective viewing window is a panoramic viewing window. 

22. (previously presented)The improved imaging apparatus of claim 14, further 
comprising a module having a memory for storing a mapping matrix. 
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23 . (original) The improved imaging apparatus of claim 22, wherein the 
module is a display/memory/local control module, 

24. (original) The improved imaging apparatus of claim 14, wherein the 
controller calculates a residual image based on a difference between a reference 
omnidirectional image and a sequential omnidirectional image to detect an anomaly and uses 
the anomaly to calculate parameters for the perspective viewing window so that the 
perspective viewing window focuses on the anomaly. 

25. (original) The improved imaging apparatus of claim 24, further 
comprising an alarm that is activated if at least a portion of the residual image exceeds a 
predetermined threshold. 

26. (original) The improved imaging apparatus of claim 14, further 
comprising an acoustic sensor coupled to the controller for detecting a sound source within 
the image scene, wherein the controller adjusts the perspective viewing window based on a 
location of the sound source. 

27. (original) The improved imaging apparatus of claim 14, further 
comprising an image transmission system for transmitting the omnidirectional image via the 
Internet. 
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28 . (original) The improved imaging apparatus of claim 27 , wherein the 
image transmission device includes a server that receives the omnidirectional image and 
transmits the omnidirectional image to at least one client. 

29 . (previously presented) The improved imaging apparatus of claim 1 4, further 

comprising: 

a remote display coupled to the image sensor; 

a first speaker and first microphone coupled to the image sensor; and 
a second speaker and second microphone coupled to the remote display, wherein the first and 
second speakers and first and second microphones form a two-way transmission link between 
the image sensor and the remote display. 

30. (previously presented) The improved imaging apparatus of claim 14, wherein 
said reflective mirror is a hyperbolic mirror having a hyperbolic cross-section. 

3 1 . (previously presented) An imaging apparatus for generating a two-dimensional 
image, comprising: 

a reflective hyperbolic miixor having a hyperbolic cross-section; 

an image sensor optically coupled to said reflective mirror that generates two- 
dimensional image data signals based on an omnidirectional image reflected by said mirror; 
and 

a controller coupled to the image sensor, wherein the controller defines a perspective 
viewing window based on configuration parameters and maps pixels from said 
omnidirectional image into said perspective viewing window; and 
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a memory for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller using a said mapping matrix to 
perform mapping of pixels from said omnidirectional image into said perspective viewing 
window. 

32 . (previously presented) The imaging apparatus of claim 3 1 , wherein the 
reflective mirror creates a one-to-one correspondence between pixels in the omnidirectional 
image and pixels in the perspective viewing window. 

33 . (previously presented) The imaging apparatus of claim 3 1 , wherein the 
controller maps the omnidirectional image to the perspective viewing window by mapping 
each pixel in the perspective viewing window with a corresponding pixel value in the 
omnidirectional image. 

34. (previously presented) The imaging apparatus of claim 14, wherein parameters 
defining the perspective viewing window include at least one of a zoom distance defined as 
the distance from the focal point of said reflective mirror to said window, a pan angle defined 
as the angle between the x axis and a line through the focal point of said reflective mirror 
perpendicular to the x-y plane and a tilt angle defined as the angle between the x-y plane and 
a vector normal to the perspective viewing window. 

3 5 . (previously presented) The imaging apparatus of claim 34, further comprising a 
user interface through which a user enters data corresponding to at least one of a desired 
zoom distance, pan angle, or tilt angle. 
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3 6. (previously presented) The imaging apparatus of claim 3 1 1 wherein the 
controller generates a mapping matrix by applying a ray tracing algorithm to each pixel in the 
perspective viewing window to determine a corresponding reflection point on the reflective 
mirror and then projecting each reflection point to a focal point of the image sensor to 
determine the corresponding location on the omnidirectional image. 

3 7. (previously presented) The imaging apparatus of claim 3 1 , wherein the 
perspective viewing window is a panoramic viewing window. 

38. (previously presented)The imaging apparatus of claim 31, wherein said 
memory contains a predeteimined mapping matrix for every set of configuration parameters 
in said parameter space. 

3 9. (previously presented) The imaging apparatus of claim 3 1 , wherein the 
controller calculates a residual image based on a difference between a reference 
omnidirectional image and a sequential omnidirectional image to detect an anomaly and uses 
the anomaly to calculate parameters for the perspective viewing window so that the 
perspective viewing window focuses on the anomaly. 

40. (previously presented) The imaging apparatus of claim 39, further comprising 
an alarm that is activated if at least a portion of the residual image exceeds a predetermined 
threshold. 
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41 . (previously presented)The imaging apparatus of claim 31, further comprising 
an acoustic sensor coupled to the controller for detecting a sound source within a scene of 
said omnidirectional image reflected by said mirror, wherein the controller adjusts the 
perspective viewing window based on a location of the sound source. 

42. (previously presented) The imaging apparatus of claim 31, further comprising 
an image transmission system for transmitting image output by said image sensor via the 
Internet. 

43 . (previously presented) The imaging apparatus of claim 3 1 , further comprising: 
a remote display coupled to the image sensor; 

a first speaker and first microphone coupled to the image sensor; and 
a second speaker and second microphone coupled to the remote display, 
wherein the first and second speakers and first and second microphones form a two- 
way transmission link between the image sensor and the remote display. 

44. (previously presented)The imaging apparatus of claim 3 1, wherein 

each said mapping matrix is configured to reduce distortion during mapping of each 
pixel in the perspective viewing window with a corresponding pixel from said 
omnidirectional image. 
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None 



None 



None 



IX. Evidence Appendix 



X. Related Proceedings Appendix 



XI. Certificate of Service 



25 



PAGE 29/29 * RCVD AT 4/25/2007 3:16:27 PM [Eastern Daylight Time] * SVR:USPTO-EFXRF-5/15 ' DNIS:2738300 ■ CSID:18015727666 * DURATION (mm-ss}:04*26 



